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ASSESSING  METHODOLOGIES  EMPLOYED  BY  U.S.  COMPANIES 
TO  ESTABLISH  BUSINESS  BENEFITS  OF  INFORMATION  SYSTEMS 


I.      BACKGROUND  AND  METHODOLOGY 


A.  Background 

The  Institute  for  Future  Technology  (IFT)  wishes  to  understand  the  use  of  methodologies 
by  U.S.  business  to  identify  the  business  benefits  ("usage  values")  of  information  systems 
(IS). 

Business  benefits  are  in  contrast  to  technical  benefits.  Technical  benefits  include  such 
factors  as:  IS  cost  savings,  processing  times,  system  quality/errors,  changes  in  system 
maintainability. 

Business  benefits  include  (but  are  not  limited  to)  such  factors  as: 
Increases  in  revenue 

Increases  in  profit  or  other  operating  measures 

Improved  responsiveness  to*customers 

Improved  customer  satisfaction 

Improved  quality  of  products  or  services 

Improved  productivity,  including  reduction  in  cycle  times 

Improved  information  (for  customers,  operating  management  and/or  executive 
management) 

IFT  wishes  to  understand  to  what  extent  individual  U.S.  companies 

Take  business  benefits  into  account  when  evaluating  or  justifying  large  systems 
investments  (either  new  systems  or  significant  modifications  to  existing  systems) 

Use  a  consistent  methodology  to  evaluate  business  benefits 

IFT  also  wants  to  understand  the  types  of  methodologies  in  use  and  the  type  of  factors  used 
for  evaluations. 
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B.  Methodology 


INPUT  collected  most  of  the  information  contained  in  this  report  from  structured 
interviews  with  U.S.  companies  with  over  $500  million  in  sales.  Information  was  obtained 
using  a  questionnaire  (shown  in  Appendix  A).  This  questionnaire  was  reviewed  with  IFT 
before  the  interviews  were  conducted. 

INPUTs  contract  with  IFT  specified  that  15  companies  were  to  be  interviewed.  In  fact,  26 
companies  were  interviewed.  Additional  companies  were  interviewed  (at  no  additional 
cost)  in  order  to  have  a  large  enough  sample  to  perform  the  additional  analysis  contained 
in  this  report. 

The  companies  interviewed  were  spread  across  major  sectors  (Exhibit  1-1).  The  actual 
companies  interviewed  are  in  Exhibit  II-2. 

The  people  interviewed  were  those  who  were  knowledgeable  about  the  planning  and 
assessment  process  in  their  organization.  Most  of  these  had  the  title  of  "Manager"  or  above 
(see  Exhibit  1-3);  in  some  organizations  the  knowledgeable  person  was  a  staff  planning 
specialist. 

The  respondents  were  forthcoming  and  answered  questions  to  the  best  of  their  knowledge. 

The  interviews  focussed  on  larger  systems  investments:  the  average  investment  cited  by 
respondents  was  over  $3  million. 

In  addition  to  the  information  obtained  from  these  interviews,  INPUT  has  also  drawn  on  its 
general  experience  and  on  other  pertinent  studies  in  making  observations  on  overall  trends 
and  planning  practices. 

A  draft  of  this  report  was  prepared  on  February  5  and  sent  to  IFT  for  review.  This  report 
incorporates  answers  to  IFT  questions  as  well  as  containing  descriptions  of  eight  referenced 
products. 
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II 


IMPORTANCE  OF  ASSESSING  BUSINESS  BENEFITS 
AND  THE  USE  OF  METHODOLOGIES 


One  of  the  key  issues  is  the  extent  to  which  organizations  are  using  (or  trying  to  use) 
business  benefits,  rather  than  technical  benefits,  to  evaluate  systems  investments.  Exhibit 
II- 1  contrasts  the  two  types. 

There  is  some  overlap  between  these  two  types  of  benefits.  For  example,  IS  cost 
savings  can  contribute  to  increased  profitability,  if  these  savings  are  large  enough. 
However,  in  most  cases,  the  IS  savings  are  not  large  enough  to  have  a  material 
effect  on  the  overall  business. 

The  major  area  where  a  purely  IS  decision  can  have  a  material  effect  on  the  rest  of 
the  business  is  where  there  is  disinvestment,  rather  than  investment.  Specifically, 
where  large  parts  of  an  IS  operation  are  outsourced  to  a  third  party  vendor  and  IS 
assets  are  sold  to  the  third  party  at  a  profit.  This  type  of  activity  is  outside  the  scope 
of  this  study  and,  more  importantly,  is  a  totally  different  sort  of  evaluation  that 
generally  only  occurs  once  in  a  period  of  many  years. 

In  order  to  judge  --  and  quantify  -  the  relative  importance  of  technical  versus  business 
benefit  evaluations  INPUT  had  each  respondent  in  an  interview  weight  the  importance  of 
each  up  to  a  total  of  100%;  this  process  is  illustrated  in  Exhibit  II-2.  Respondents  were 
asked  to  rate  their  organizations  for  1992  and  make  an  estimate  for  1997. 

Exhibit  II-3  summarizes  the  results  of  these  rankings: 

Five  out  of  25  respondents  saw  business  benefits  as  accounting  for  over  90%  of  the 
evaluation  activities  in  their  firm. 

An  additional  10  companies  put  the  business  benefit  ranking  in  the  70-85% 
category. 

Only  6  companies  rated  business  benefits  under  50% 

The  importance  of  business  benefits  is  seen  by  respondents  as  becoming  slightly 
more  important  over  the  next  five  years 


In  INPUTs  experience  this  represents  a  considerable  change  from  ten  years  ago,  when 
significant  systems  investments  were  far  more  likely  to  be  evaluated  almost  wholly  on 
technical  issues. 
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Based  on  other  INPUT  research,  these  changes  in  evaluation  methodologies  are  consistent 
with  other  changes  going  on  in  IS: 

There  is  more  decentralization  generally  in  U.S.  organizations. 

Related  to  the  above,  central  IS  organizations  have  considerably  less  power  and 
influence  than  in  the  past. 

Centrally-controlled  IS  budgets  are,  in  general,  only  increasing  at  the  rate  of 
inflation;  some  are  falling  in  absolute  terms. 

End  user  or  functional  departments  are  much  more  active  in  making  decisions  on 
applications. 

"Pure  technology"  investments  are  generally  out  of  favor;  systems  investments  must 
been  seen  as  having  a  positive  impact  on  the  business. 

As  Exhibit  II-4  shows,  about  one-quarter  of  the  respondents  still  use  technical  evaluation  as 
their  predominant  evaluation  method.  However,  over  half  of  respondents  are  using 
business  benefits  as  the  predominant  means  of  evaluation.  About  one-quarter  of 
companies  do  not  use  a  methodology  at  all  but  respond  to  each  situation  on  an  ad  hoc  basis 
(that  is,  they  deal  with  each  situation  as  it  arises). 

As  noted  on  Page  1  of  this  report,  this  report  does  not  discuss  or  analyze  the  way  in  which 
technical  benefits  are  evaluated.  Technical  evaluations  are  very  dependent  on  software 
tools  which  analyze  computer  and  network  performance.  As  additional  information  to  IFT, 
Appendix  B  contains  information  on  products  from  one  of  the  best  known  software 
vendors,  BGS,  which  supplies  technical  evaluation  software. 

Even  where  methodologies  are  used,  only  a  minority  of  companies  now  use  a  single, 
consistent  methodology  (Exhibit  II-5). 

A  single,  consistent  methodology  is  far  more  likely  to  be  used  as  part  of  a  financial  model 
than  any  other  type  of  methodology  (Exhibit  II-6).  As  will  be  discussed  later  in  the  report, 
there  are  advantages  to  using  variable  methodologies. 
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III.    BUSINESS  BENEFIT  EVALUATION 


The  overwhelming  majority  of  respondents  have  found  business  benefit  evaluations  useful 
(Exhibit  III-l). 

Even  where  an  evaluation  can  only  deal  with  part  of  the  business  benefit,  the 
process  still  provides  much  useful  information. 

In  some  companies  the  analytic  process  that  is  gone  through  is  just  as  useful  as  the 
results  of  the  process. 


Exhibit  III-2  summarizes  the  relative  importance  of  the  different  kinds  of  business  benefits, 
by  showing  the  percentage  of  respondents  that  rated  the  benefit  a  "4"  or  a  "5"  on  a  scale  of 
5.  Every  respondent  gave  a  rating  for  each  one  of  the  items  listed  under  "Potential 
Business  Benefit".  For  example,  81%  of  the  respondents  rated  "Increased  productivity"  as  a 
"4"  or  a  "5"  on  a  scale  of  5.  The  other  19%  of  respondents  gave  a  rating  of  "1",  "2"  or  "3". 

Interestingly,  the  three  lowest  ranked  are  those  that  are  the  easiest  to  measure.  This  is 
analyzed  immediately  below. 

Some  benefits,  such  as  revenue  or  market  share,  are  easy  to  measure.  Others,  that  may  be 
equally  important,  such  as  improved  information  or  improved  customer  satisfaction  may  be 
much  more  difficult  to  measure  satisfactorily. 

On  the  other  hand,  the  impact  of  IS^nvestments  may  be  direct  or  indirect.  That  is, 

A  direct  benefit  of  an  IS  investment  may  be  lower  inventory  levels  that  can  be 
traced  accurately  to  IS  changes. 

On  the  other  hand,  increased  revenues  are  often  the  result  of  many  initiatives  within 
a  firm.  It  may  be  very  difficult  to  attribute  a  particular  portion  of  this  solely  to  IS 
activities.  However,  these  indirect  benefits  may  be  very  important  also. 

The  responses  were  further  analyzed  to  see  if  there  were  any  other  factors  mentioned. 
There  were  other  items  brought  up  but  they  were  not  applicable: 

A  few  respondents  reintroduced  technical  factors,  which  had  been  intentionally 
excluded  from  the  analysis. 

Some  respondents  went  into  more  detail  on  what  attributes  were  being  measured 
(e.g.,  number  of  prescriptions  filled). 

A  few  other  respondents  gave  general  responses  such  as  "supporting  business 
objectives"  after  they  had  already  provided  more  informative  detailed  answers. 
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It  is  not  surprising  that  there  were  no  additional  categories  of  criteria  volunteered.  The 
choices  provided  in  the  questionnaire  were  very  inclusive. 

Exhibit  III-3  is  a  matrix  showing  the  four  possible  combinations  of  relationships  described 
above.  Each  "cell"  in  the  matrix  is  analyzed  in  more  detail  in  Exhibits  m-4  through  Exhibit 
III-7. 

Each  of  these  exhibits  analyzes  and  discusses  the  major  types  of  business  benefits 
(which  are  underlined)  within  each  group. 

As  can  be  seen,  important  business  benefits  fall  within  each  group. 
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The  importance  of  a  particular  business  benefit  (or  a  series  of  benefits)  may  vary 
significantly,  even  within  the  same  company: 

Corporate  goals  and  objectives  may  change  (e.g.,  from  market  share  to  profitability). 

Different  user  departments  often  have  different  objectives  (or  place  different 
weights  on  the  same  series  of  objectives). 

There  may  be  a  series  of  evaluations  to  pass. 

For  example,  a  systems  investment  may  have  to  first  prove  that  it  can  pass  a 
hurdle  for  a  financial  rate  of  return. 

After  that  has  been  achieved,  then  other,  more  qualitative  factors,  may  be 
used  to  give  the  investment  a  priority. 

Large  systems  projects  must  always  be  evaluated  as  to  their  financial  rate  of  return.  This  ' 
may  be  within  the  evaluation  of  an  overall  change  in  function,  i.e.,  changes  to  the 
manufacturing  or  distribution  process,  where  the  IS  investment  is  one  of  several  investment 
components.  Smaller  projects  may  not  be  evaluated  in  financial  terms  especially  if  a 
change  is  considered  to  be  unavoidable.  Examples: 

Changes  to  regulations  or  government  reporting  requirements  which  are  mandated 
by  law 

Errors  in  software  which  must  be  fixed  to  enable  an  application  to  continue  to 
support  the  business  unit.  These  are  generally  small  on  an  individual  basis,  but  very 
large  when  added  up  in  the  course  of  a  year;  the  typical  U.S.  company  spends  over 
half  of  its  budget  for  IS  personnel  on  this  type  of  activity.  Much  of  this  is 
considered  to  be  a  cost  of  doing  business  and  is  not  separately  evaluated.  This  was 
the  reason  why  INPUT  generally  limited  its  analysis  to  "new  systems  or  significant 
modifications  to  existing  systems"  (page  1  of  the  report). 
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IV.    METHODOLOGIES  USED  TO  ESTABLISH  BUSINESS  BENEFITS 


A.      Methodologies  Used  to  Assess  IS  Investments 


INPUT  has  used  the  classification  format  shown  in  Exhibit  III-4  to  classify  the  business 
benefits  identified  in  Exhibit  III-2.  The  result  is  shown  in  Exhibit  IV- 1. 

Only  two  benefits,  productivity  and  cost  savings  are  both  direct  and  measurable. 

The  other  benefits  are  indirect  and/or  difficult  to  measure.  As  noted  earlier,  these 
can  be  very  important  benefits,  but  evaluating  them  in  a  consistent  way  can  be 
difficult. 

Note:  In  financial  analysis,  the  terms  "tangible"  and  "intangible"  benefits  are  commonly 
used. 

'Tangible"  is  equivalent  to  "measurable". 

"Intangible"  is  equivalent  to  "difficult  to  measure". 

A  financial  model  is  fairly  straightforward.  Exhibit  IV-3  describes  the  mechanisms 
involved.  The  most  difficult  aspect  of  the  financial  model  is  to  accurately  quantify  the 
benefits  involved: 

Are  the  benefits  in  fact  direct^ 

How  likely  are  the  benefits  to  actually  occur? 

How  long  should  benefits  be  counted  as  flowing  from  IS  initiatives,  as  opposed  to 
being  driven  by  continued  general  improvements  to  business  practices? 

Quantification  of  benefits  can  be  assisted  by  such  devices  as  market  surveys  and  data  bases 
of  historic  performance.  However,  there  is  always  a  large  element  of  judgment  involved. 

The  financial  models  used  fall  into  these  categories: 

Commercially-supplied  financial  planning  software  packages.  Examples  of  these  are 
shown  in  Exhibit  IV-4. 

Internally-developed  software. 
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All  of  the  financial  models  are  driven  by  the  mechanism  described  in  Exhibit  IV-3  —  The 
organizations  financial  records  (organized  by  a  "chart  of  accounts"  in  a  "general  ledger"  is 
passed  into  the  financial  planning  system. 

All  the  models  allow  for  different  types  of  financial  and  statistical  processes  to  be  applied 
to  different  simulations  and  assumptions.  Financial  tests  can  include  the  following: 

Present  value  ($) 
Future  value  ($) 

Internal  rate  of  return  (expressed  as  a  %) 
Payback  (expressed  in  months  or  years) 

A  company  generally  only  uses  one  type  of  test.  They  are  all  driving  by  the  same 
parameters  and  generally  are  measuring  the  time  value  of  money. 

All  the  software  packages  are  driven  off  the  same  general  theory,  described  above.  The 
main  differences  are: 

The  complexity  of  the  organization  they  are  aimed  at.  Some  companies  have  many 
departments  and  many  layers  of  organizations.  Companies  like  that  place  a  high 
value  on  the  ability  of  a  model  to  consolidate  results  from  many  intermediate  units, 
with  different  currencies,  etc.  These  complex  models  will  offer  a  wealth  of  financial 
and  statistical  functions  to  be  used  by  sophisticated  financial  analysts. 

Smaller  projects,  whose  effects  are  limited  to  a  self-contained  organizational,  will 
often  use  a  less  sophisticated  model  in  an  easier-to-use  software  package. 

The  output  of  models  can  be  expressed  in  dollars  (and/or  other  currency  units),  percent 
returns  and/or  time  to  payback.  As  mentioned  above,  these  measures  as  essentially 
interchangeable,  although  financial  planners  may  have  very  strong  opinions  on  the 
advantages  and  disadvantages  of  each. 

Cost/benefit  analysis  are  driven  by  the  particular  situation.  Unlike  financial  models,  there 
are  not  standard,  commercially-available  products  that  can  be  purchased.  This  is  because 
having  an  understanding  of  the  business  issues  is  more  important  than  the  particular 
methodology  employed. 

Exhibit  IV-5  is  a  key  to  detailed  product  information  contained  in  appendixes.  Note  that 
there  are  demonstration  diskettes  for  three  of  these  products. 
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U.S.  businesses  often  employ  a  consulting  firm  to  perform  these  kinds  of  cost/benefit 
analyses  when  evaluating  systems  investments.  Major  firms  involved  in  this  type  of  activity 
include: 

Andersen  Consulting 
Booz  Allen 

Digital  Equipment  (consulting  division) 
IBM  (consulting  division) 
McKinsey  &  Co. 
Price  Waterhouse 

None  of  these  firms  has  a  single  methodology  that  is  used  to  assess  costs  and  benefits, 
although  all  use  financial  models  to  capture  direct/measurable  benefits. 

Their  strengths  are  having  an  in-depth  knowledge  of  an  industry  and  being  able  to 
apply  lessons  learned  from  other  companies  in  the  same  industry  as  a  client  for 
whom  they  are  performing  the  evaluation. 

They  view  their  collection  of  project  information  has  highly  proprietary. 


One  consulting  firm,  DMR,  represents  are  partial  exception  to  this  practice.  Up  to  now, 
much  of  their  systems  evaluation  activity  has  been  similar  in  nature  to  the  activities  of  the 
consulting  organizations  listed  above.  However,  they  are  in  the  process  of  developing  a 
methodology  to  show  the  broader  impact  of  systems  on  business.  Additional  information 
on  DMR's  approach,  "Productivity  Plus",  as  well  as  on  the  firm  in  general  is  in  Appendix  I. 

B.      Process  for  Evaluating  Business  Investments  Generally 


The  process  for  evaluating  business  investments  generally  is  very  important  to 
understanding  systems  investments:  That  is  because  systems  investments  are  increasingly 
viewed  as  being  integral  to  overall  business  investments. 

The  evaluation  of  business  investments  is  generally  part  of  a  strategic  or  business  planning 
process.  The  relationship  of  a  systems  plan  to  the  larger  business  plan  is  shown  in  Exhibit 
IV-6.  The  IS  organization  is  rarely  the  initiator  of  these  strategic  consulting  assignments 
(Exhibit  IV-7).  INPUTs  research  indicates  that  large  systems  projects  are  almost  always 
preceded  by  this  type  of  planning  (Exhibit  IV-8). 

The  planning  involved  will  almost  always  use  a  financial  model  as  one  of  the  analytic 
components.  However,  the  cost/benefit  approach  is  the  more  important  component. 
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Exhibit  1-1 


Sectors  Interviewed 


Manufacturing/Distribution  8 


Financial  Services  8 


Government/Education 


Other  Services 


26 


v  
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Exhibit  1-2 


Organizations  Interviewed 


Blue  Cross/Blue  Shield  of 
Maryland 


Carlson  Companies  -  Hospitality 
Div. 


CUNA  Mutual  Insurance 


First  Wisconsin  Trust 


Genetech 


Gold  Kist,  Inc. 


Indiana  Depart  of  Labor 


Johnson  &  Johnson  -  Research 
Division 


Marine  Midland  Bank 


Marshall  &  llsley  Corp.  (Banking) 
Medical  Center,  Inc. 
Michigan  State  University 
Missouri  Public  Safety  Div. 


Monsanto 


Mutual  of  Omaha  Insurance 


Nissan  Motors  USA 


Northern  Telecom 


Providence  Hospital 


Provident  National  Bank 


Prudential  Insurance 


Texas  Tech 


Turner  Broadcasting 


U.S.  Air  Force  -  McGill  AFB 


University  of  Michigan 
Walgreen  Corp. 


Warner  Lambert  Squibb 
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Exhibit  1-3 


Titles  of  Respondents 


Percent  of 

Title  Respondents 


Vice  President  27% 


Director  12% 


Manager  42% 


Staff  Planning  Specialist  19% 


100% 
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Exhibit  1-4 


Size  of  System-Related  Investments 


Average 

Investment  Size  Percent 


Under  $1  million  35% 


$1  -  4  million  35% 


$5  million  and  over  30% 


Average  (mean):  $3.5  million 


Exhibit  11-1 


System  Investment  Evaluation  Criteria 


Category 


Example 


Technical  benefits 


IS  cost  savings 

Improved  processing  times 

Improving  system  quality 


Business  benefits 


Revenue  or  profit  increases 

Improved  customer  service 
or  satisfaction 

Product  quality 

Improved  productivity 
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Exhibit  11-2 


Survey  Methodology  for  Allocating 
Importance  of  Evaluation  Categories 


Evaluation  Category  Relative  Importance 
Technical  benefits  X% 


Business  benefits  Y% 


1 00% 
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Exhibit  11-3 

Importance  of  Evaluating 
Business  Benefits:  1992  and  1997 

Number  of 

'     .     .       .  Respondents 
Relative  Importance 

of  Business  Benefits  1992  1997 

0-15%  0  0 

20  -  45%  6  4 

50  -  65%  4  5 

70-85%  10  10 

90  -  1 00%  5  6 


Total*  25  25 


One  respondent  did  not  answer  this  question. 
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Exhibit  11-4 


Type  of  Evaluation  Methodology  Used 


Predominant  Percent  of 

Evaluation  Focus  Companies 

Business  benefits 

Business  analysis  23% 
(cost/benefit) 

Financial  model  31% 

Subtotal  54% 

Technical  23% 

None/ad  hoc  23% 


100% 


c 

Exhibit  11-5 


Extent  of  Use  of  a  Single/Consistent 

Methodology 


Type  of  Methodology         Percent  of  Respondents 


Methodology  is  used 

Single/consistent  31% 
Variable  46% 


Methodology  is  not  qsed  23% 


1 00% 


'  INPUT 
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Exhibit  11-6 

Focus  of  Evaluation  and  Type 
of  Methodology 

Percent  of  Companies  Using 

Single  Methodology 
Focus  of  *  w 

Evaluation  Yes       No     Method.  Total 

Business  benefits 

.  Business  analysis  4%      19%  23% 

(cost/benefit) 
.  Financial  model    23%        8%  31% 

.  Subtotal  27%      27%  54% 


Technical  40/0      19%  23% 


None/ad  hoc  23%  23% 


Total  31o/o      46%      23%  100% 


Exhibit  111-1 


Usefulness  of  Business 
Benefit  Evaluation 


Is  Business  Benefit  Percent  of 

Evaluation  Useful?  Respondents 


Yes  69% 

No  4% 

Don't  know/not  sure  8% 

Not  applicable  19% 
(not  doing  it) 

100% 
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Exhibit  111-2 


Relative  Importance  of  Business 
Benefits  Used  to  Evaluate 
Systems  Investments 


Potential  Percent  Rating 

Business  Benefit  Benefit  Important 

Increased  productivity  81% 

Improved  responsiveness  to  customers  81% 

Improved  customer  satisfaction  77% 

Improved  information  77% 

Improved  product/service  quality  69% 

Improved  profit/cost  savings  50% 

Increased  revenue  42% 

Increased  market  share  31% 


Rated  4  or  5  on  a  scale  of  5 


Exhibit  111-3 


Categories  of  Business  Benefits 


Direct 


Type  of  Benefit 


IS  Impact  Measurable 


Measurable 
benefits-  all  or 
most  attributable 
to  IS  initiatives 


Difficult 
to  Measure 

IS  impact  is  large 
but  intangible 


See  Exhibit  111-4 
for  examples 


See  Exhibit  111-6 
for  examples 


Indirect 


Measurable 
benefits  but 
only  part  can 
be  attributable  to 
IS  initiatives 


Benefits  may  be 
very  hard  to 
measure;  only 
part  of  benefits 
is  attributable 
to  IS 


See  Exhibit  111-5 
for  examples 


See  Exhibit  111-7 
for  examples 
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Exhibit  lli-4 


Business  Benefit  Examples: 

Type  of  Benefit:  Measurable 
Impact  of  IS:  Direct 


Reduced  user  department  costs  where  the  decrease  can  be  tracked  to  an  IS 
change  (new  or  modified  software;  more  efficient  hardware  or 
communications).  Examples: 

Lower  inventory  levels,  lowering  working  capital  and  interest 
expenses 

Reduction  in  functional  department  personnel  requirements  as  a 
result  of  system  improvements.  (This  is  in  addition  to  any  direct 
savings  within  the  IS  operation  -  this  type  of  benefit  is  classified  as  a 
technical,  not  a  business,  benefit.) 

Less  wastage/rework  due  to  IS  changes.  (Note:  This  type  of  benefit 
often  belongs  in  the  "indirect"  category  because  an  IS  initiative  is 
often  only  one  component  in  a  series  of  changes.) 

This  type  of  benefit  is  often  very  easy  to  reduce  to  financial  terms  and 
to  include  in  a  financial  model. 


Improved  customer  service  that  is  primarily  a  result  of  an  IS  initiative. 
Examples: 

Improved  transaction  cycle  time  (e.g.,  real  time  data  base  access 
compared  to  overnight  updates) 

Improved  response  time  to  customer  queries. 

Fewer  out  of  stock  situations  for  customer  orders.  This  could  be 
verified  as  increasing  revenues;  however,  in  this  type  of  situation 
orders  may  only  be  delayed  or  different  items  may  be  ordered  - 
consequently,  considerable  analysis  may  be  required  to  properly 
allocated  benefits. 
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Business  Benefit  Examples: 

Type  of  Benefit:  Measurable 
Impact  of  IS:  Indirect 


Increased  revenue.  Most  revenue  increases  are  difficult  to  trace  back  to 
specific  IS  initiatives.  It  is  sometimes  difficult  to  allocate  a  portion  of  a 
revenue  improvement  to  IS,  where  IS  is  one  part  of  a  series  of  changes. 

However,  the  IS  role  is  often  crucial.  When  a  functional  department  directly 
controls  IS  resources  or  IS  budgets,  the  department  will  include  IS  changes 
as  a  line  item  in  an  overall  improvement  plan.  In  this  type  of  very  common 
situation,  IS  expenditures  are  part  of  an  overall  plan  -  the  IS  expense,  even 
if  large,  may  only  be  5-10%  of  the  overall  plan. 

Increased  market  share.  Some  companies  are  driven  primarily  by  market 
share,  rather  than  revenue  increases.  The  comments  made  under  "revenue" 
above  apply  to  market  share  also. 

Improved  product  or  service  quality  -  when  part  of  a  company-wide  quality 
program  that  emphasizes  quality  measurement.  [Otherwise  this  benefit  is  in 
Exhibit  III-8,  "Indirect  -  Difficult  to  Measure"] 

Reduced  expenditures  over  a  broad  category.  An  example  of  this  is 
"business  process  re-engineering",  where  a  company  reexamines  all  of  its 
business  processes  and  changes  or  eliminates  many  of  them.  IS  changes 
are  often  a  critical  elements  in  making  the  larger  changes  possible; 
however,  it  is  often  very  difficult  to  measure  that  the  IS  contribution  is. 

Improved  customer  service,  where  IS  is  one  of  several  initiatives.  An 
example  is  an  improved  customer  order  process,  where  IS  is  the  backbone 
of  changes,  but  many  others  are  taking  place  at  the  same  time. 

Benefits  and  changes  in  this  category  can  be  measured  and  often  can  have 
firm  financial  benefits  attached  to  the  changes. 

The  entire  program  of  changes  (IS  and  non-IS)  can  be  costed  and 
financial  benefits  estimated.  These  financial  effects  can  be  put  into  a 
financial  model. 

Trying  to  isolate  the  IS  component  involves  arbitrary  allocation  of 
benefits.  This  produces  technical  problems  in  allocation.  More 
importantly,  it  produces  political  problems  since  the  different 
departments  involved  may  each  want  a  majority  of  benefit  allocated 
to  their  own  contribution. 
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Business  Benefit  Examples: 

Type  of  Benefit:  Difficult  to  Measure 
Impact  of  IS:  Direct 


Improved  information  to  internal  departments  and  management. 

This  can  be  a  real  benefit,  but  is  difficult  to  measure  and  almost 
impossible  to  place  a  financial  value  on.  An  example  is  a  "customer 
information  system",  where  all  the  activity  for  a  particular  customer  is 
linked  together. 

The  justification  for  many  new  applications  often  includes  "better 
information"  routinely,  whether  or  not  it  is  believed  to  be  true 
(because  it  is  difficult  to  prove  one  way  or  the  other).  This  has  tended 
to  devalue  a  potentially  important  benefit. 


Improved  satisfaction  by  internal  users  of  IS  services.  In  the  mid-1980s  this 
was  a  very  important  measure  used  by  many  central  IS  departments. 
Periodic  surveys  were  used  to  determine  customer  satisfaction.  However, 
internal  satisfaction  as  a  benefit  is  much  less  important  now: 

The  survey  results  have  proved  difficult  to  compare  over  time. 
Satisfaction  levels  rise  or  fall  for  reasons,  that  on  investigation,  prove 
to  be  caused  by  the  surveying  process. 

More  importantly,  high  or  low  satisfaction  can  be  caused  by  many 
factors.  It  is  more  important  to  identify  the  other  factors  and  measure 
them  separately. 
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Business  Benefit  Examples: 

Type  of  Benefit:  Difficult  to  Measure 
Impact  of  IS:  Indirect 


Many  people,  both  inside  and  outside  of  IS,  often  disregard  this  category, 
since  even  after  going  to  the  difficulty  of  trying  to  measure  the  benefit,  the 
impact  of  the  IS  contribution  is  still  indirect. 


However,  the  factors  involved  may  be  part  of  some  companies'  core  culture 
or  may  be  critical  to  a  company's  long  term  success.  These  types  of 
initiatives  include: 

Helping  a  company  become  more  knowledgeable  about  -  and  more 
res po ns i ve  to  ~  its  customers'  needs . 

Improving  product/service  quality  (where  there  is  not  an  established 
mechanism  for  measuring  quality). 

Improving  customer  satisfaction  in  targeted  areas. 


The  IS  commitment  to  these  types  of  programs  may  be  large;  however, 
even  if  these  types  of  programs  are  successful,  tracking  the  IS  contribution 
may  be  impossible. 


These  kinds  of  projects  are  often  initiated  outside  of  the  IS  planning 
process  and  the  IS  component  is  specified  after  the  overall  plan  has  been 
approved. 
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Classification  of  Business  Benefits 


Type  of  Benefit 


Direct 


Indirect 


IS  Impact  Measurable 


Productivity 
Cost  savings 


Productivity 
Quality 
Cost  savings 
Revenue 


Market  share 


Difficult 
to  Measure 

.  Responsiveness 

.  Satisfaction 
(internal) 

•  Information 


.  Responsiveness 

•  Satisfaction 
(external) 

•  Quality 
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Applicability  of  Tools/Methodologies 


Type  of  Benefit 


IS  Impact  Measurable 


Difficult 
to  Measure 


Direct 


Financial  model  Cost/benefit 


Indirect 


Cost/benefit 


(Usually  not 
initiated  by  IS; 
part  of  general 
business 
planning) 
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Exhibit  IV-3 

Financial  Model:  Mechanism 

Expected  incremental  expense  by  month 
or  quarter 

Hardware 

Software  packages 
Internal  personnel 
Communications 

External  services  (from  professional 
services  or  systems  integration  vendors) 

Expected  cost  savings  or  revenue  increases 
directly  traceable  to  IS  by  month  or  quarter 

Five  to  seven  year  time  horizon 

Investment  must  pass  standard  corporate  tests. 
For  example, 

Discounted  cash  flow  (e.g.,  positive  value 
after  five  years  at  a  15%  discount  rate) 
or 

Payback  of  investment  after  X  years 


YNIFT 


INPUT 


Exhibit  IV-4 


Financial  Model:  Examples  of  Sources 

Very  powerful  financial  planning  software 
packages  (usually  used  by  financial  planners) 
Examples: 

Commander  FDC  (Comshare) 

Fastar,  Hyperion  and  Micro  Control  (IMRS) 

Javelin  (Information  Resources) 

Less  complex  financial  planning  software 
packages.  Examples: 

-     LUCRAWIZ  (via  Software  Publishing) 
Encore  Plus  (Ferox  Microsystems) 
CF:  Cash  Flow  Analysis  (Superior  Software) 

In-house  developed  systems  that  may  be  very 
complex  or  spreadsheet-driven 
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Additional 


Financial  Model  Information 


Financial  See  Demonstration 

Model  Vendor       Appendix  Diskette 


Comshare 
IMRS 

Information 
Resources 

Via  Software 
Publishing 


Ferox 


G 


Superior 
Software 


H 


X 
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Exhibit  IV-6 


Strategic  Planning/Consulting  Preceding 
the  Systems  Implementation  Process 


Activity 


Identify  business 
opportunity/problem 


I 


Form  team  for  analysis 
and  recommendations 


Produce  business  plan 


I 


Prepare  overview 
systems 
implementation  plan 


Prepare  other 
implementation  plans 


t 

!  Disconnect 


Group(s) 
Involved 

Executive  management 
User  management 
Strategic  consultant 
Internal  consultant 

Ad  hoc  (internal) 
Internal  consultant 
Strategic  consultant 


Strategic  consultant 
Systems  consultant 
IS 


To  Phase  B 
(Implementation) 
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Exhibit  IV-7 


Percentage  of  Strategic 

Consulting  Assignments  Initiated 

by  Different  Types  of  Units 

/  Executive 

/  Mgmt. 

/  20% 

Central  IS  /  ^T^^ 

User  Depts.  / 

\                 (Including  their  IS)  / 

75%  / 

Exhibit  IV-8 


Relationship  of  Planning  and  Implementation 
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TEND  IX  A 


METHODOLOGIES  USED  TO  EVALUATE 
LARGE  SYSTEMS  INVESTMENTS 


INPUT,  an  international  research  and  consulting  firm,  is  conducting  a  study  on  the 
methodologies  used  by  firms  to  justify  large  systems-related  investments.  Neither  the 
identity  of  you  or  your  firm  will  be  published.  In  return  for  your  taking  part  in  our  study  we 
will  supply  you  with  a  summary  of  the  study's  findings  at  no  charge. 


1.       In  the  last  three  years,  what  were  the  largest  systems-related  investments  that  were 
made  (type  of  project,  dollar  amount)?  [This  can  be  hardware,  software,  outside 
services  or  some  combination.] 


2.       What  was  the  chief  justification  for  each  of  these?  [If  more  than  five,  Umit  question 
to  top  five  investments.] 


3.       Was  there  a  single  methodology  used  to  evaluate  systems  investments? 
Yes  No 


Why? 


If  yes,  please  describe  the  methodology  briefly 
[and  then  go  to  question  ] 
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If  no,  are  there  several  methodologies  in  use? 

Yes    No   

Why? 


If  yes,  please  describe. 


4.       In  evaluating  systems  investments  there  are  two  general  types  of  criteria  that  are 

used,  technical  benefits  and  business  benefits.  [Use  definitions  below  if  necessary.] 

Technical  benefits  examples:  IS  cost  savings,  improved  processing  times, 
improving  system  quality,  etc. 

Business  benefits  examples:  Revenue  or  profit  increases,  improved  customer 
service  or  satisfaction,  improved  productivity  or  product  quality. 


In  making  investment  evaluations,  what  is  the  split  in  the  importance  between 
technical  and  business  benefits  (allocating  the  two  between  100%)?  What  was  this 
split  5  years  ago?  What  do  you  expect  it  to  be  in  5  years? 


1992  1987  1997 

Technical       

Business     

100%  100%  100% 

What  is  causing  this  change? 
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Which  business  benefits  are  most  important  in  evaluating  systems  investments? 
[Check  off  and  add  to  bottom,  if  not  on  the  list]. 


How  important  are  these  other  factors  on  a  scale  of  1  to  5,  with  5  being  most 
important?  [Read  factors  from  list  that  were  not  checked  off.] 

Most  Imp.  Rating 

Revenue     

Profit     

Market  Share     

Responsiveness  to  customers     

Customer  satisfaction     

Product/service  quality     

Productivity     

Improved  information  (to 

management  or  customers)     

Other    

Other     
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For  the  factors  that  were  checked  off  or  received  a  rating  of  4  or  5:  You  rated  as  being  important. 
How  does  your  firm  measure  these  benefits  now?  What  improvements  are  you  planning  to  make? 

Factor  Measurement  Improvement 


Revenue 


Profit 


Market  share 


Responsiveness  to 
customers 


Customer  satisfaction 


Product/service  quality 


Productivity 


Improved  information 
(to  management  or 
customers) 


Other 


Other 
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7. 


What  type  of  tools  or  formal  methodologies  does  your  firm  use  in  making 
evaluations  or  justifications  for  major  systems  investments? 


If  a  formal  methodology  exists:  Could  you  supply  us  with  a  copy  (with  your 
company's  identification  omitted,  if  you  wish)? 


8.       Overall,  how  useful  has  your  firm  found  using  business  benefit  evaluations  for  major 
systems  investments? 


What  changes  are  being  made  or  should  be  made  to  the  process? 


9.       What  advice  would  you  give  a  company  that  was  planning  to  use  business  benefits  as 
a  basis  for  evaluating  systems  investments? 


Thank  you  for  participating  in  our  survey. 
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Appendix  B 


PUT  Questionnaire 


dy  Title: 


CONFIDENTIAL 

.Project  Code/Catalog  No.    CO  DDDDDQQ 

□□□ 


96 


of  Interview: 


□  Vendor    □  Telephone 

□  User       □  On-Site 

□  Other      □  Mail 


Company: 
Address: 


Interviewer  Initials 
Interview  Date 
QC  Initials 
QC  Date 
Data  Entry  Initials 
Data  Entry  Date 


Comp 


#  Employees: 


1  II 
1  II 

III 
III 

I  III' 

JLJIL 

II  1 

m 

II  1 

1  II 

III 

ii  in 

II  1 

i  ii 

II  1 

1  II 

III 

ii  in 

li  1 

ty/State/Zip: 
•lain  Phone: 

pnndPTif.(sV^ 


FAX  # 


Name 


Title 


Phone/Ext. 


irrals: 


istry  (User  Interviews  Only): 

□  Discrete  Mfg. 

□  Process  Mfg. 

□  Transportation 

□  Utilities 

□  Communications 

□  Retail 


□  Wholesale 

□  Banking/Finance 

□  Insurance 

□  Medical 

□  Services 

□  Education 


□  Federal  Government 

□  State  &  Local  Government 

□  Consumer/Home 

□  Other  Industry  Specific 

□  Cross-Industry 


iments: 


RES2SO/CM 
3/90 


International  IT  Intelligence  Services 


Clients  make  informed  decisions  more  quickly  and  economically  by  using  INPUT'S 
services.  Since  1974,  information  technology  (IT)  users  and  vendors  throughout  the 
world  have  relied  on  INPUT  for  data,  research,  objective  analysis  and  insightful 
opinions  to  prepare  their  plans,  market  assessments  and  business  directions, 
particularly  in  computer  software  and  services. 

Contact  us  today  to  learn  how  your  company  can  use  INPUT'S  knowledge  and 
experience  to  grow  and  profit  in  the  revolutionary  IT  world  of  the  1990s. 


Subscription  Services 


Information  Services  Markets 

-  Worldwide  and  country  data 

-  Vertical  industry  analysis 

Business  Integration  Markets 

Client/Server  Applications  and 
Directions 

Client/Server  Software 

Outsourcing  Markets 

Information  Services  Vendor 
Profiles  and  Analysis 

EDI/Electronic  Commerce 

US.  Federal  Government  IT 
Markets 

IT  Customer  Services  Directions 
(Europe) 

Service  Features 


Research-based  reports  on  trends,  etc. 
(Over  100  in-depth  reports  a  year) 

Frequent  bulletins  on  events,  issues, 
etc. 

5-year  market  forecasts 

Competitive  analysis 

Access  to  experienced  consultants 

Immediate  answers  to  questions 

On-site  presentations 

Annual  conference 


Databases 


•  Software  and  Services  Market 
Forecasts 

•  Software  and  Services  Vendors 

•  U.S.  Federal  Government 

-  Procurement  Plans  (PAR) 

-  Forecasts 

-  Awards  (FAIT) 

•  Commercial  Application  (LEADS) 

Custom  Projects 


For  Vendors — analyze: 

•  Market  strategies  and  tactics 

•  Product/service  opportunities 

•  Customer  satisfaction  levels 

•  Competitive  positioning 

•  Acquisition  targets 

For  Buyers — evaluate: 

•  Specific  vendor  capabilities 

•  Outsourcing  options 

•  Systems  plans 

•  Peer  position 

Other  Services 


Acqiusition/paitnership  searches 


INPUT  Worldwide 

Frankfurt 

SudetenstraBe  9 
D-35428  Langgons- 
Niederkleen 
Germany 

Tel.  +49  (0)  6447-7229 
Fax  +49  (0)  6447-7327 

London 

17  Hill  Street 
London  W1X  7FB 
England 

Tel.  +44  (0)  71  493-9335 
Fax +44  (0)71  629-0179 

New  York 

400  Frank  W.  Burr  Blvd. 
Teaneck,NJ  07666 
U.S.A. 

Tel.  1  (201)  801-0050 
Fax  1  (201)  801-0441 

Paris 

24,  avenue  du  Recteur 
Poincare' 
75016  Paris 
France 

Tel.  +33  (1)  46  47  65  65 
Fax +33  (1)46  47  69  50 

San  Francisco 
1881  Landings  Drive 
Mountain  View 
CA  94043-0848 
U.S.A. 

Tel.  1  (415)961-3300 
Fax  1  (415)  961-3966 

Tokyo 

Saida  Building,  4-6, 
Kanda  Sakuma-cho 
Chiyoda-ku,  Tokyo  101 
Japan 

Tel. +81  3  3864-0531 
Fax +81  3  3864-4114 

Washington,  D.C. 

1953  Gallows  Road 
Suite  560 

Vienna,  VA  22182 
U.S.A. 

Tel.  1  (703)  847-6870 
Fax  1  (703)  847-6872 
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